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ABSTRACT 


Introduction: the reference and counter-reference system (RCR) defines a hierarchical 
flow of assistance to patients and/or communication among professionals. The refer- 
ence is a document that requires consulting and/or forwards patients to other doctors 
(consultants) who draw up a reply document, the counter-reference. In Brazil, the RCR 
system does not work well, and no study has systematically evaluated it. Objectives: to 
evaluate reference reports from health services from Belo Horizonte (PBH), municipali- 
ties in the metropolitan region of Belo Horizonte (RMBH), and the General Hospital from 
UFMG (HC) elaborated by general practitioners (GP) or specialists (ESP) and received 
in the Endocrinology Service from the GH, and define the necessary information in 
reference to enable the consultant to decide for a face-to-face consultation. The doctors 
who work in the Family Health Program (FHP) or secondary and tertiary care doctors 
were considered GPs. Methods: this was an observational and sectional study. Results: 
two researchers (k = 0.652) evaluated 282 reports and identified that 62% could have 
been assisted in the primary care. The frequency of reports with enough information 

for decision-making was greater in those from the GH and elaborated by specialists. A 
greater number of reports were accepted when presenting endocrinology diseases and 
physical and complementary examination data. Conclusions: most of the referred pa- 
tients presented disorders that could have received care in the primary care; a reference 
report must contain a diagnosis and data from additional tests and physical examination. 


Key words: Primary Health Care; Referral and Consultation; Academic Medical Cen- 
ters; Consultants. 


RESUMO 


Introdução: o sistema de referência e contrarreferência (RCR) define um fluxo hierarquizado 
de atenção aos pacientes e/ou de comunicação entre profissionais. A referência é um docu- 
mento que demanda uma consultoria e/ou encaminha pacientes a outros médicos (consul- 
tores), que elaboram um documento-resposta, a contrarreferência. No Brasil, o RCR não fun- 
ciona bem e não há estudos que o avaliem sistematicamente. Objetivos: avaliar relatórios 
de referência provenientes de serviços de saúde da Prefeitura de Belo Horizonte (PBH), de 
municípios da região metropolitana de Belo Horizonte (RMBH) e do Hospital das Clínicas da 
UFMG (HC), elaborados por clínicos gerais (CLN) ou especialistas (ESP), recebidos no Ser- 
viço de Endocrinologia do HC, e definir informações necessárias em referência que permitam 
ao consultor decidir por interconsulta presencial. Foram considerados CLNs os médicos que 
atuam no Programa Saúde da Família (PSF) ou clínicos da atenção secundária e terciária. 
Métodos: trata-se de estudo seccional e observacional. Resultados: dois pesquisadores 
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(R=0,652) avaliaram 282 relatórios, sendo constatado 
que 62% poderiam ser abordados na atenção primária. A 
frequência de relatórios com informações suficientes para 
a tomada de decisão foi maior naqueles oriundos do HC e 
elaborados por especialistas. Maior número de relatórios 
foi aceito quando apresentaram doenças próprias da 
Endocrinologia, dados do exame físico e dos exames 
complementares. Conclusões: a maioria dos pacientes 
referidos apresentava transtornos abordáveis na AP e um 
relatório de referência deve conter diagnóstico, dados dos 
exames complementares e do exame físico. 


Palavras-chave: Atenção Primária à Saúde; Referência 
e Consulta; Centros Médicos Acadêmicos; Consultores. 


INTRODUCTION 


The lack of health services assisting the popula- 
tion is a global problem that is compounded in coun- 
tries like Brazil, given the scarcity of resources and 
absence of effective traffic of information. The refer- 
ence and counter-reference system (RCR), an inter- 
face between the various levels of health care, has a 
prominent position in any health care system, public 
or private, with implications for patient care, perma- 
nent education, and financial cost. 

In Brazil, the Unified Health System (SUS) has 
been extended to the entire Brazilian population 
since the enactment of the Federal Constitution of 
1988 under public funding. The system was orga- 
nized to be decentralized, hierarchical, and with pri- 
mary care (AP) as its main gateway like those in other 
countries such as England, Canada, and Austrália! 
Thus, the definition of needs in patient care and use 
of available resources is the responsibility of the AP 
doctor. It is expected that he is able to drive most of 
the nosological context prevalent in the community 
without the need for other levels of care. 

The secondary care (AS), performed by special- 
ists, should be understood as an assistant care to the 
AP. In it, patients are always referred by a physician. 
Access to AS must be filtered by the need and sever- 
ity of cases, and not the interests of the physician 
requesting the reference. In AS, the approach tends 
to be fragmented, focused on diseases, organs, or 
pathogenetic mechanisms. The professional is able 
to deal with more specific and/or complex problems, 
including those unusual. 

The RCR system can be an effective tool for promot- 
ing permanent education. The proper counter-reference 
is the formative training and seems the most practical 
way that can be used because it is a feedback factor 
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linked to the concrete activity.*º It allows a continuous 
and never-ending learning process inserted in the pro- 
fessional practice, such as in healthcare, in general, and 
particularly in those working in AP, with little time avail- 
able for education activities and classroom training. 

In Brazil, the RCR system does not work well, and 
there are no studies systematically assessing it. The ex- 
cess demand makes it impossible to schedule first con- 
sultations with specialists, it is also observed that many 
of them can be performed by general practitioners, 
and because of this,” the Special Service for Diagno- 
sis and Treatment in Endocrinology and Metabolism 
(SEEM) of the General Hospital (HC) from UFMG has 
developed a system of distant ambulatory consultation 
since 1998 that provides customers with a hierarchy of 
instruments for interconsultations: cell phone or land- 
line, fax and email, in addition to consolidating the tra- 
ditional referral system for medical consultations. 

The RCR system starts with the preparation of a 
reference report (RR), which in Belo Horizonte, is 
generally written and delivered to the patient, who is 
responsible for presenting it to the service to which 
he has been referred. However, there is much dis- 
agreement between the consultant doctors and those 
consulting about the ideal content of the RR.:”º Any- 
way, the improvement of RRs and communication be- 
tween AP doctors and specialists can avoid repeating 
diagnostic investigations, dissatisfaction, and loss of 
the confidence of patients in the medical practice.! 

Therefore, the aim of this study was to evaluate RRs 
sent to the SEEM from the HC-UFMG and set the pa- 
rameters for the decision making by the consultant in 
addition to understanding the quality of RRs establish- 
ing the profiles of customers, consultants, and patients. 


MATERIAL AND METHODS 


A total of 282 reference reports received by the 
SEEM HC-UFMG in the period from January 1 to June 
30, 2004, coming from the health care services of the 
Municipality of Belo Horizonte (PBH), municipalities 
in the metropolitan region of Belo Horizonte (MRBH), 
and HC were analyzed. That number corresponded 
to all the reports received during the period. This was 
a quantitative cross-sectional design study. 

RRs were analyzed by two researchers according 
to the established protocol for this research, consid- 
ering the suggestions of Crossley and White.” The 
evaluation started with the general description of 
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the data on institutional origin, gender, specialty, 
and time since graduation for the consultant doc- 
tor; request reason; readability; use of standardized 
forms; information: clinical history, physical exami- 
nation, complementary tests, diagnosis, treatment, 
planning, severity, psychiatric disorders, socioeco- 
nomic status, and adhesion; patient age; and re- 
quest destination (sufficiency for the decision and 
acceptance to schedule). The request destination 
had three alternatives: schedule for endocrinology 
(acceptance), non-scheduled (refusal), and insuffi- 
cient (in need of more information). Subsequently, 
requesting groups set up according to the institu- 
tional origin were compared — HC, PBH, and RMBH 
— and specialty of the consultant doctor — general 
practitioners (CLN) and specialists (ESP). 

Doctors working in AP in the basic units ofthe Fam- 
ily Health Program (PSF), in clinical medicine and in- 
ternal medicine, were considered as CLNs. Surgeons, 
Gynecologists, and others who work in AS were con- 
sidered ESPs. 

Three groups were defined regarding the diagno- 
sis: (A) consisting of problems that can be dealt with- 
in AP; (B) consisting of endocrinology disorders; (C) 
consisting of disorders still with no definite diagnosis. 

Group B include the disorders that CLN referred 
to as: thyroid nodules; thyrotoxicosis; thyroid can- 
cer; diabetes mellitus type 1 (DMI); diseases of the 
hypothalamus and pituitary; diseases of the adrenal 
glands; secondary hypertension (endocrine); gonadal 
dysfunction; hirsutism and hyperandrogenism; gyne- 
comastia; disorders of sexual differentiation; puberty 
endocrine disorders; disorders of mineral metabo- 
lism and bone metabolic diseases; severe and refrac- 
tory dyslipidemia; hypoglycemia (non-related to DM), 
and endocrine disorders in pregnancy. 

Group A consisted of: primary hypothyroidism, 
diffuse goiter, chronic thyroiditis, pre-diabetes, diabe- 
tes mellitus type 2 (DM2), dyslipidemia, and isolated 
obesity. Regarding DM 2, we highlight that there are 
situations considered suitable for evaluation by the 
endocrinologist: uncontrolled DM 2 with rapid weight 
loss (days to a few weeks); doubts as to the best phar- 
macological approaches to DM 2, especially insulin 
and new drugs (or those lacking experimental data); 
and DM 2 evolving with complications. 

It is noteworthy that these criteria were developed 
by endocrinologists who also act as AP clinicians. Doc- 
tors who prepared the RRs and those evaluating the 
request will be referred to consulting and consultants, 


respectively. The scheduled RRs or referred to another 

sector were considered sufficient for making decisions. 
The rejected RRs were: 

= with problems that could be dealt at AP (group A) 
and without evidence of a specific need from the 
consultant doctor; 

= without a definite diagnosis (group €C) and no evi- 
dence of having an endocrine disorder, especially 
in the absence of severity; 

= with endocrinology problems (group B) already 
solved. 


Some RRs in group C were pending when evi- 
dence of severity was nonexistent, and the data sug- 
gested endocrine disease. The pending status means 
waiting for further information to be reassessed. 

This study was approved by the Ethics Committee 
of UFMG, and the statistical analysis was performed us- 
ing SPSS 12.0 for Windows. The analysis of differences 
between groups (HC, PBH, and RMBH) and specialties 
(CLN and ESP) was performed using the chi-square 
test (X2), with a significance level of 5% (p <0.05 ). 

The K index (Kappa) was calculated to analyze 
concordance between the decision taken by the two re- 
searchers based on the data obtained in the pilot test in 
SPSS. The following classifications were used: K <0.20, 
poor concordance; 0.21 to 0.40, regular concordance; 
0.41 to 0.60, moderate concordance; 0.61 to 0.80, good 
concordance; and 0.81 to 1.00, optimal concordance.! 

Multivariate analysis was performed using the bi- 
nary logistic regression model to obtain the adjusted 
measure of the influence of various factors affecting 
the variables on the consultant's decision. The binary 
logistic regression model was built considering the suf- 
ficiency for decision and scheduling acceptance as 
dependent variables. Reports that were accepted or 
declined were considered sufficient for a decision. The 
selection of independent variables was performed us- 
ing the stepwise method with the forward conditional 
selection (entry criterion and conditional probability). 


RESULTS 


Overall description of reference reports 
Out of the consulting doctors who prepared the 
RRs, 65% came from the HC, 59% were men, 65% were 


specialists, and 51% had more than nine years since 
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graduation. Most RRs were prepared in standardized 
forms (95%), legible (98%), and contained the re- 
questing reason (97%). It was observed that 78% of 
RRs presented at most two information for diagnosis 
(81%), clinical history (29%), complementary tests 
(27%), treatments (24%), and physical examination 
(11%). It was found that 62% (n = 248) of RRs present- 
ed diseases that could be dealt at AP. After the con- 
sultant's analysis, 50% were accepted for scheduling 
and 77% were deemed sufficient for a decision. 


Reference reports according to 
institutional origin (HC, PBH, RMBH) 


RRs separated by institutional origin were dif- 
ferent in the following parameters: specialty and 
time since graduation for consultant doctors, use of 
a standardized form, information number, and re- 
quest's destination. 

RRs derived from HC (versus non-HC) were 
mainly prepared by specialists (88% vs 19%; Xº = 
107.4, p <0.001) and by doctors with less than nine 
years since graduation (56% vs 39%; Xº = 8.9, p = 
0.003), written in standardized forms (98% vs 89%; 
Xº = 12.7, p <0.001), and often unscheduled (38% 
versus 8%; X? = 27.3, p <0.001). The presence of four 
or more information data was more frequent in RRs 
from PBH (45%) when compared to the other institu- 
tions (17%) (Xº = 19.63, p <0.001). 


Reference reports according to 
information sufficiency for decision-making 


The institutional origin, specialty, and diagnos- 
tic group (A, B, or C) were significantly associated 
with information sufficiency for decision makins. 
The frequency of sufficient RRs was higher in those 


from HC (84% vs 63% non-HC; X2 = 14.9, p <0.001) and 
produced by ESP (90% vs. 54% non-ESP HC; Xº= 374 
p <0.001). When the analysis was conducted sepa- 
rately by institutional origin, this difference (CLN vs. 
ESP) remained only in HC (93% vs. 28% ESP CLN; 
Xº = 50.9; p <0.001). The diagnostic group B showed 
higher frequency of sufficient RRs than groups A and 
C (87% vs. 72%; Xº = 8.05; p = 0.005). The diagnostic 
group C (still undiagnosed disorders) did not pres- 
ent the highest frequency of insufficient RRs. When 
analyzed separately by institutional origin, this differ- 
ence (group B vs. non-B) remained only on non-HC 
RRs (Xº = 13.4, p <0.001). 

Regarding information, the sufficiency was also 
influenced by physical examination (93% vs 75%; OR 
= 6.87, IC (95%) = 1.24 to 70.00, Xº = 6.9, p = 0.009), 
and complementary exams (85% vs 74%; OR = 2.56, IC 
(95%) = 1.08 to 4.71; Xº = 4.9, p= 0.027). 

The binary logistic regression analysis showed 
that specialty, diagnostic group, complementary ex- 
ams, and readability were independent factors for 
information sufficiency in the consultant's decision 
(Table 1, model 1). 


Reference reports according to 
the specialty of the consultant doctor 


RRs from non-HC CLN were more frequent than 
those from HC (81% vs 12%, Xº= 108.7, p <0.00001), 
featuring the most often clinical history (44% vs 31%, 
Xº= 4.0; p = 0.045), diagnosis (38% vs 19%, Xº= 5.7, p 
= 0.017), treatment (53% vs 28%, X2= 12.5; p = 0.0004), 
and three or more information (56% versus 28%, Xº= 
15.9; p = 0.0001). 

The RRs CLN were less sufficient for the decision 
drawn up by ESPs (24% vs 71%, X2 = 38.7; p <0.00001). 
No difference was observed on acceptance of RRs, 
however, the refusal rate was higher in those prepared 
by CLN (vs ESP) (46% vs 2%; X2 =37.5, p <0.00001). 


Table 1 - Multivariate analysis of factors that influence sufficiency 


Specialty(CLN) -2.928 
Readability 2.606 
Model 1 
Complementary tests 1.344 
Diagnostic group (B) 2133 
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0.053 0.019 — 0.151 <0.001 
13.541 1.093 — 167.7 0.042 
3.833 1.192 12.331 0.024 
8.441 2.366 — 30.114 0.001 
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Decision concordance between researchers 


The decision on the request destination was car- 
ried out by two researchers and compared with the 
K index; the value of 0.652 (SPSS) was found, which 
indicates that there was good concordance. 


DISCUSSION 


Under the endocrinology consultant's perspec- 
tive in public and university services, it was observed 
that most patients referred to endocrinology (62%) 
had disorders that could be addressed by the clini- 
cian (CLN) in the primary care (AP). This study also 
suggests that, to evaluate the relevance of referrals, 
the reference report (RR) must be legible and con- 
tain the following information: consultant's specialty, 
diagnosis, most relevant data of the clinical history, 
physical examination, and complementary tests. RRs 
made by specialists and coming from the university 
hospital seemed more than enough for the consul- 
tant's decision-makinsg. 

This study showed that most of the references con- 
tained endocrine disorders that are part of the body 
of knowledge for AP, such as type 2 diabetes, dyslip- 
idemia, obesity, and primary hypothyroidism. Elwyn 
and Stott! retrospectively evaluated 168 RRs issued 
by a single general practitioner to several specialists 
and concluded that 35% of RRs could be preventable, 
based on available resources and skills of the doctors 
involved. Jenkins! concluded that 13% of 234 RRs sent 
to several specialists were inadequate, particularly 
those made by clinical specialists. Donohoe et al”, in 
a prospective study with general practitioners (con- 
sulting doctors) and specialists (consultants), found 
that 30% of the references were inappropriate, with a 
resolution equal to that made by general practitioners. 

The higher frequency of inappropriate RRs or 
those that could be dealt by AP in this study (62%), 
compared with other studies, can be partially ex- 
plained by the difference in the definition herein pre- 
sented of what is appropriate. The analysis was based 
on what constitutes endocrinology disorders from the 
perspective of a specialist. This competence was not 
analyzed under the perspective of the consulting or 
requesting physician. In any case, it is emphasized 
that the two researchers in this study are doctors who 
act as endocrinologists and general practitioners 
(general clinicians). 


Other studies comment that there are also other 
factors that could potentially influence the reference 
adequacy, such as temporal aspects, if late or early; 
size and complexity of the problem; solving capacity”, 
quality or consulting/communication between the 
clinician and specialist; medical and legal aspects 
such as patient's interests or needs; different perspec- 
tives of the specialist, clinical, patient, or system ad- 
ministrator; assessment of the problem evolution;! 
existence and quality of protocols; secondary or 
marginal aspects such as readability and insufficient 
information, especially the motive of the consulting 
doctor. This fact may be related to the characteristics 
of the health system in Brazil, highlighting the diffi- 
culties faced by AP doctors such as an excessive de- 
mand of patients, limited access to complementary 
propaedeutics, and lack of support, which increase 
their chance to practice defensive medicine with an 
excessive use of available resources, including refer- 
ral to secondary care (AS). 

Anyway, a dialogue between those involved in the 
RCR system interface stands out as needed. Therefore, 
this study shows that the concordance between the re- 
searchers was good (k=(0.652), however, in the context 
of assessing RRs by teachers in the university service, 
a higher concordance should be expected. This can 
be partly explained by the profile of the researchers. 
One is a clinical endocrinologist acting in teaching, 
research, and extension in an outpatient endocrinol- 
ogy clinic; the other is a clinician advised in his mas- 
ter's degree by the first doctor, working in the internal 
medicine clinic, teaching, in endocrinology research, 
and his professional training. These profiles, amid a 
more flexible definition of refusal, generated a wide 
variation in the researcher's decision. 

This study analyzes the instrument that initiates 
the process of communication between AP and AS, 
the RRs. Gandhi et al.” and Wright et al.” affirm that 
RRs plays a critical role in an effective communica- 
tion, and it is also important for the quality of care. 

Depending on the objective limitation of meeting 
the excessive demand of patients, it was essential to 
investigate what information should compose an RR 
in order to allow the decision by the RCR consultant 
where the request for a specialist consultation under- 
goes medical and administrative evaluation consider- 
ing relevance and priority levels. It is noteworthy that 
the analysis of the information that should compose 
an RR for non-face-to-face consulting purposes was 
not the objective of this study because RRs had a 
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strong component of “transfer” of a patient to another 
service and not of an actual consultation, be it con- 
ventional or not. 

Consideration should be given to a consultant role 
being influenced by the SEEM-HC's role in the pro- 
cess of developing a non-face-to-face outpatient con- 
sulting system focused on continuous medical educa- 
tion. Thus, inadequate specialist referrals could have 
a “not accepted” destination and those from CLN 
could be more often allocated as pending by asking 
additional information as a strategy to establish a 
communication channel. The influence ofthe consul- 
tant's specialty in the information sufficiency for the 
decision remained only in RRs from HC when institu- 
tional origin was analyzed separately. Probably, RRs 
from HC CLN were less accepted and more insuffi- 
cient (dependent on additional information) due to 
the greater involvement within the university campus 
and medical training (graduate and residency). 

No controversy about the importance of the RR 
and its capacity for effective RCR was observed. In- 
deed, Grol et al.8, studying 637 RRs from general 
practitioners in AP, observed suitability of RRs by 
specialists when they had four or more information 
data, particularly clinical history, physical examina- 
tion findings, complementary investigations, and pre- 
vious or current treatment institution. 

Regarding RR contents and comparing those with 
reports from other authors,!'!$ data from clinical his- 
tory and physical examination were less frequent in 
this study; no difference in the frequency of other 
contents was observed. Importantly, in all studies, 
90% or more of RRs had at least one type of informa- 
tion such as clinical history, diagnosis,2! and re- 
questing reason. These differences can be attributed 
to the characteristics of consultants, specialties, use 
of a standardized printed form, nosological complex- 
ity, and the institutional origin of the consultant. 

Itis important to consider that the mere presence 
of the diagnosis did not influence the decision, the 
group (A, B and €) to which the case belonged did. 
Thus, one can infer that an RR with endocrine disor- 
ders in group A, that could be dealt at AP, and group 
C, undiagnosed, should contain other information to 
justify the referral. The same was observed by Linné? 
whose analysis of RRs referred to a specialist service 
in obesity showed insufficient information, suggest- 
ing as the explanation the fact that many consultant 
doctors considered the obesity diagnosis enough to 
justify the referral. 
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In this study, the diagnostic group B was more ac- 
cepted for scheduling and less refused or allocated as 
pending. The analysis of this aspect by Kvame et al.? 
suggests the need for a joint definition of the field of 
action based on a committee analysis for every spe- 
cialty with AP representatives to better guide physi- 
cians in interconsultation requests. It also proposes 
the need for changing attitude on both sides, suggest- 
ing that the patient care be achieved in a joint effort.” 

With regard exclusively to the number and the infor- 
mation, unrelated to the synthesis that means the consul- 
tant's decision, this study showed that reports from PBH 
or prepared by CLN were more complete, with more in- 
formation and more often with data for clinical history, 
treatment, physical examination, and complementary 
tests. This probably reflects the fact that CLNs (most RRs 
coming from PBH) tend to more integrated and with 
comprehensive approaches seeking better resolution of 
patient problems. This contrast can not be explained by 
the difference in complexity or type of disease because 
the groups' profiles (A, B, and C), and those of endo- 
crine disorders (data not shown), were similar. 

The parameters that RRs must contain for decision 
making were not clearly defined. However, some con- 
siderations can be made. As expected, the diagnostic 
group was associated with information sufficiency for 
the decision. Also, some data from the physical exam- 
ination and complementary tests probably influenced 
because this information tend to be more objective. 
The institutional origin and consultant's specialty was 
significantly associated with information sufficiency. 
This does not mean that the evaluating doctor consid- 
ered these parameters in the decision but that some 
characteristics of consulting doctors are contained in 
them. For example, in educational institutions, there 
is a tendency to keep patients internally, under the 
attention of specialty services, regardless of the prob- 
lem's complexity. In fact, HC RRs were more often not 
scheduled. There is an overlap between institutional 
origin and specialty, shown by the very high correla- 
tion between them. One could consider that readabili- 
ty influenced the researchers' decision. Although logi- 
cal, this was not a problem because the readability 
was high. Moreover, and most likely because of this, 
the data from this study regarding readability were 
weak, with very a wide confidence interval and its 
lower limit close to OR near to 1. 

As for patients, Preston et al.” reported that many 
feel as if they had been left in the “limbo” when sent 
to different sectors of the health system. 
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CONCLUSIONS 


This study showed that a significant fraction of re- 
ferrals to a clinical specialty, such as endocrinology, 
could be avoided. It is suggested that this could be 
done with an improved communication between AP 
and AS. To effect a pact of criteria defining diseases 
that can be dealt at AP and specialties is essential, that 
is, the role of the medical field in AP and AS should 
be better defined reducing uncertainties in this inter- 
face. The reports should also be improved, not only 
those for reference but also those for counter-refer- 
ence. This study suggests, for example, that diagnos- 
tic and clinical data from the physical examination 
and complementary tests are important. It is impor- 
tant that the physicians involved further enhance the 
RCR system. Hence, they should recognize that the 
RCR, besides its assistance-oriented character, can 
be a powerful instrument of permanent education. 
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